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Description 

This invention disclosure describes a graphics adapter with frame-buffer compression engine and a network 
interface. A network attached graphics adapter is a device similar to a traditional graphics adapter that adds to 
or replaces the standard video output port or ports, such as a DVI port, with a network interface port or ports 
such as an Ethernet or Infmiband port. The network interface port can be used for transmitting the contents of 
the frame-buffer in either compressed or uncompressed form or any region of the graphics adapter's memory to 
one or more remote network attached devices such as a compute node with a network interface card. This 
capability allows multiple users to remotely access and view graphics content created by graphics accelerators 
in compute nodes, such as high-availabUity (HA) servers, located anywhere, such as in a "back room", "glass 
room" or an "Internet Data Center (IDC)", using existing network infrastructure and display devices, such as 
"thin clients** or "appliances**. 

Current display refresh on the graphics adapter works as follows: 

While (TRUE) 

If start of vertical retrace 

For all pixels in frame-buffer 

Read pixel from frame-buffer 
Convert pixel to output format 
Transmit out analog/digital port 

The functionality listed above would be replaced with the following: 

While (TRUE) 

If new update is requested a time threshold has been reached 

If there has been a change in some portion of the frame-buffer 
If compression of the frame-buffer enabled 
Compress the frame-buffer 

Convert the compressed buffer to Internet Protocol (IP) or other network protocol 
Transmit the compressed frame-buffer out the network interface. 

Else 

Convert the frame-buffer to IP or other network protocol 

Transmit the uncompressed frame-buffer out the network interface port. 

The algorithm listed above could be combined with algorithms presented in the invention disclosure titled 
"Algorithms for removing redundant data in an Internet Visualization Appliance" which involved imposing a 
tiling on the image and keeping track of the tiles that have been modified. 

The algorithm listed above could also be modified slightly to handle capabilities such as overlay planes and 
window id planes and cursor. In this scenario, multiple regions of the frame-buffer would be simultaneously 
read, for example the primary display surface and overlay plane. From this data the displayed image would be 
assembled and compressed by the frame-buffer compression engine. The following algorithm demonstrates this 
functionality. 

While (TRUE) 

If new update is requested a time threshold has been reached 

If there has been a change in some portion of the primary display surface or overlay planes 
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If compression is enabled 

Coalesce and compress the buffer resulting from the primary and overlay planes 
And cursor location. 

Convert the compressed buffer to Internet Protocol (IP) or other network protocol 
Transmit the compressed frame-buffer out the network interface. 

Else 

Convert the coalesced buffer to IP or other network protocol 
Transmit the uncompressed buffer out the network interface port. 

The operation of coalescing involves reading the pixel from the higher priority buffer or determining the buffer 
that has precedence and performing color LUT conversions and also merging in the cursor. 

One of the disadvantages of current display refresh technology is that the data that is sent out the frame-buffer is 
not tagged to allow delivery of the frame-buffer contents to multiple destinations. Consequently, graphics 
adapters are usually not used in "server" like environments where multiple users share a single machine. As a 
result, technologies like Citrix Metaframe are used that send the graphics protocol over the network which must 
be rendered by the client. 

There have been graphics cards created with multiple video output ports. With these types of video cards it 
might be possible to have one user using one region of the frame-buffer and another user using another region 
of the frame-buffer. It may also be possible to load a computer with more than one graphics adapters and have a 
one-to-one mapping between users and graphics adapters. However, the number of physical ports on the 
graphics adapter and number of graphics cards in the system limits these approaches. 

Using this invention it becomes possible for multiple users/clients to share or time-slice a graphics card and 
have their own unique session and/or desktop. Given a graphics adapter with it's own video memory associate a 
unique region of the graphics adapter memory with each unique network device. For example, if a graphics 
adapter has 32 Mbytes of memory the first 4 Mbytes might be assigned to network device 1, the second 4 
Mbytes may be assigned to network device 2, and so on. The algorithm listed below would then be used to 
update each network-attached device: 

While (TRUE) 

If new update is requested by a specific network device or a time threshold has been reached 
If the frame-buffer for the specific network device has been modified 

Configure the frame-buffer compression engine to compress the frame-buffer 

for the network device. 
Compress the frame-buffer for the network device. 
Convert to IP 

Configure the network interface unit to send the compressed frame-buffer to the 
network device. 

Transmit the compressed frame-buffer to the network device out the network 
Interface port. 

Another possible use of this invention is in the area of collaboration where multiple users share a common 
desktop or window within a frame-buffer. In this scenario the same frame-buffer or same region of the frame- 
buffer would be compressed and transmitted to multiple network-attached devices. 



dacted 



Advantages over Prior Art 

• Off-loads the compression, read back, and transmittal of the frame-buffer from the host system or CPU. 
Currently, to transfer an image rendered in a graphics adapter to a remote network attached device the 
frame-buffer would be required to be read back from the graphics card into system memory, 
compressed, and sent over the host system's network interface. Alternatively, a PCI board or appliance 
could be used as identified in the invention disclosure called "Internet Graphics Appliance". However, 
this invention has additional benefits beyond the IGA approach, which are listed below. 

• Transmittal of the contents of the frame-buffer only occurs when the contents of the frame-buffer 
change. Current graphics adapters transmit the entire contents of the frame-buffer to the monitor at a 
continuous refresh rate independent of whether or not any pixels within the frame-buffer have changed 
in value. This is necessary because monitors and other display devices do not have their own local 
display refresh memory. As a result, the overhead of display refresh imposes a constant drain on the 
graphics adapters memory sub-system, which significantly impacts graphics rendering performance 
since the same memory is being accessed by the rendering engine and display refresh unit. This 
invention decouples the actual display device refresh from the graphics adapter which could free up 
significant memory bandwidth when actual frame-buffer image updates are occurring less often than the 
video refresh rate of the display device. It is noted that if a network fabric such as Ethernet were used for 
display refresh, the capability presented here would be dependent on the remote network attached 
monitor or display device to be able to persist the transmitted frame-buffer in its own local memory and 
to refresh from a local memory source. Currently this technology does not exist and this is the subject of 
another invention disclosure that is in the process of being submitted. 

• Transmittal of the contents of the frame-buffer can be limited to regions of interest. Current video output 
ports on graphics card transmit the entire contents of the frame-buffer at a constant monitor refresh rate 
such as 75 Hz independent of the portions of the frame-buffer that have been modified. Since the 
graphics adapter renders the pixels into the frame-buffer it should be able to easily keep track of the 
extents of the changes or tiles or regions that have changed within the frame-buffer very easily. The 
graphics adapter needs to only transmit the regions that have changed. 

• Transmittal of frame-buffer contents or some other region of the graphics adapters local memory could 
be tagged/addressed and sent to more than one network attached device. This would allow multiple users 
of a graphics card to be supported simultaneously with their own session or in a collaborative 
environment. Current video output ports on graphics cards transmit the contents of the frame-buffer out 
the video port to whatever is plugged into it. The contents of the frame-buffer are not tagged and 
transmitted to a destination. This makes it very difficult to support multiple users or multiple display 
devices simultaneously using the same graphics adapter. 

• Lower system cost and higher efficiency can be obtained in comparison to the invention disclosure 
"Internet Graphics Appliance" (IGA) or PCI board solution. The IGA is a separate appliance that grabs 
the incoming frame-buffer at refresh rates and performs many operations on the incoming pixel stream 
to identify if the current frame has changed and the regions or tiles that have changed. These changes 
could be more easily and efficiently tracked by a graphics adapter and could be used to identify when 
data in the frame-buffer needs to be transmitted and the regions that are required to be transmitted. 
Additionally, a separate device with it's own frame-buffer memory and electrical components would not 
be required which would further reduce the overall system cost. It is also noted that this capability does 
not currently exist within a graphics adapter. The invention disclosure titled "Internet Graphics 
Appliance" presents a methodology that would allow the frame-buffer or region of the frame-buffer to 
be set over the network to a remote display based on technologies and approaches currently available. 
The IGA invention could be viewed as a technology that could be used while transitioning from current 
technologies to the approach identified in this invention. 
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• Graphics adapter frame-buffer can be viewed remotely anytime, anyplace, and anywhere with no 
distance limitations. Current video out technologies has distance limitations and cannot be viewed from 
any network-attached device. 

• Graphics Adapter with Frame-Buffer Compression Engine and Network Interface. 



This section describes possible embodiments of the invention: 

• The frame-buffer compression engine and networking functionality may be implemented in a separate 
chip or chips on the graphics adapter as shown in Figure 1 Variation 1. The frame-buffer compression 
engine and networking functionality may be implemented in a custom ASIC chip, DSP, FPGA, or other 
alternative. It may be possible to easily design a graphics board with this type of functionality on it 
without modifying the currently available graphics chips. 

• The frame-buffer compression engine and networking functionality may be implemented as a block or 
blocks within the graphics chip as shown in Figure 2 Variation 2. In this design scenario current 
graphics chips would have to be modified significantly to add this type of functionality. 

• The video and DVI port may be eliminated from the graphics adapter completely as shown in Figure 3 
Variation 3. This would reduce the cost of the graphics adapter. 

• The PCI or AGP interface may be eliminated from the graphics adapter in which case the graphics 
commands themselves would be received from the network interface as shown in Figure 4 Variation 4. 
However, the network connection would have to be significantly faster than what is currently available. 

• The PCI or AGP interface could be substituted with another type of network interface such as 
Infinniband or Fiber Channel or another type of network technology that may become available. This 
embodiment is not shown in a diagram, but is easily created by substituting in the diagrams everywhere 
where is states PCI or AGP. 

• Graphics Adapter with Frame-Buffer Compression Engine Only: In this embodiment the graphics 
adapter only has the frame-buffer compression engine and does not have a network interface. In this 
embodiment the graphics card compresses the frame-buffer. The compressed frame-buffer would be sent 
back to the host system and transmitted over the network by the host to the client. 
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Possible Embodiments 
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Figure 1 Variation 1 




Network Attachable Graphics Adapter 



Network Attachable Graphics Chip 
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Figure 2 Variation 2 
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Network Attachable Graphics Adapter 



Network Attachable Graphics Chip 



Standard Graphics Chipset Core 



Frame-Buffer Compression 
Engine 



Network Interface Unit 




Figure 3 Variation 3 



Network Attachable Graphics Appliance 



Network Attachable Graphics Chip 



Standard Graphics Chipset Core 
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Network Interface Unit 




Figure 4 Variation 4 
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4000 Fountain Place 
1445 ross avenue 
Dallas, Texas 75202-2790 

telephone (214) 855-7500 
facsimile (214) 855-7534 

website www.munsch.com 

writer's direct dial 



214 855-7519 
agupta@munsch.com 
Direct Fax No. 214-978-5304 

Redacted 

Mr. Roland Hochmuth 
Hewlett-Packard Company 
3404 E. Harmony Road 
Fort Collins, CO 80527 

Re: U.S. Patent Application 

Entitled: SYSTEM AND METHOD FOR 
DISPLAYING AN IMAGE ON A DISPLAY DEVICE 
Your Ref. No.: 10017760-1 
Our File No.: 5804.57-1 

U.S. Patent Application 

Entitled: SYSTEM AND METHOD FOR COMMUNICATING 
GRAPHICS IMAGE DATA OVER A COMMUNICATION NETWORK 
Your Ref. No.: 10017761-1 
Our File No.: 5804.58 

Dear Roland: 

Enclosed are draft copies of the above-identified applications for patent along with a copy of the 
drawings. Please coordinate with all other inventors of each patent application for their careful review to 
see that the documents accurately and adequately describe the invention. After the review, please fax 
or e-mail any changes or corrections to me. 

Please note that the inventor has a duty to disclose material prior art to the PTO. Such prior art 
includes relevant patents and printed publications, information concerning public use of methods or 
apparatus related to the invention, and information on public use or sales of the invention (or related 
methods or apparatus) made more than a year ago. Failure to disclose such prior art may invalidate 
any patent issuing on the application. 

Should you have any questions or comments, please do not hesitate to contact me. 

Sincerely, 

Anand Gupta 

AG:vc 
Enclosures 

cc: L.Joy Griebenow (via facsimile w/attachments) 
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